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Abstract: Various notions of fluctuations exist depending on the way one chooses 
to measure them. We discuss two extreme cases (continuous measurement versus 
long inter-measurement times) and we see their relation with entropy production 
and with escape rates. A simple explanation of why the relative entropy satisfies a 
Hamilton- Jacobi equation is added. 

1. Introduction 

AU statistical mechanics originates from a fluctuation theory. For 
equilibrium statistical mechanics, there is the theory pioneered by Boltz- 
mann, Planck and Einstein where probabilities are derived from macro- 
scopic conditions. So then, if we know say the values of the energy, the 
particle number and perhaps a few other macroscopic quantities, we can 
compute specific heats, susceptibilities, etc. and see how they relate to 
fluctuations in energy, magnetization, etc. Moreover, thermodynamic 
potentials appear in the equilibrium fluctuation law to weigh the devi- 
ation from the macroscopic condition. 

One would hope to achieve the same for nonequilibrium systems. Things 
are however more complicated here. To reach a nonequilibrium steady 
state, the system must be sufficiently open to an external world that 
permits the flow of energy or particles. As a consequence, there is 
no immediate analogue of the microcanonical ensemble from which all 
other equilibrium ensembles are derived. True, we could include that 
external world in our system to have again a closed total system but 
then we must deal with the problem of relaxation to equilibrium with 
special and long-lived nonequilibrium constraints. 

In the present note, we deal with aspects of nonequilibrium fluc- 
tuations that, while clearly important, appear less known. One issue 
concerns the very notion of fluctuation. We emphasize an operational 
interpretation in which the nature of the fluctuation in fact depends on 
how we measure them. Secondly, we point out a couple of relations be- 
tween fluctuation rates and thermodynamic or kinetic quantities. We 
explain the relation between relative entropy and entropy production 
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and we suggest a new way of computing escape rates in nonequilibrium 
systems. 

2. Fluctuations 

Fluctuations can be large or small, steady or time-dependent; they 
obviously depend on the observable and they can e.g. measure devia- 
tions around the law of large numbers for a macroscopic quantity. Yet, 
in all events, it takes time to measure fluctuations. But if we speak 
about time-averages, we also need to specify how much time we leave 
after every observation. Especially so where time-correlations matter, 
we can expect to see relevant differences between more static (with 
large inter-times) and dynamic (with small inter-times) fluctuations. 

Imagine we have a time-homogeneous steady state regime and ask 
what is the probability to see a particular value for some observable. 
E.g. for given nonequilibrium constraints, you ask how plausible it is 
to see a particular (local) density profile. That will be reflected in the 
frequency of occurence. You will probably want to make time-averages 
over a period T that is sufficiently long to see a steady value. Yet, 
for finite T there are always fluctuations. The point is now that these 
depend on the way of recording. To be more precise, think of a process 
{Xt,t > 0), where Xt is the state of your system at time t (position, 
local density, magnetization, local kinetic energy....). Wc flx r > 0; by 
a time-average we mean the average of X^, X2r, X^^-, . . . , Xt- Over a 
long time T, we see the value x with a fraction 

, r?i 

#(^) = TttE^^-- (2-1) 

I T I k=l 

[Sa=x = 1 when a = X and is zero otherwise.) That is the relative 
frequency to observe x when measuring with time-intervals of length 
T > 0, for a duration T. There will appear two extremes: either r J, 
is very small, or t ^ T. The flrst type of fluctuations will be called 
dynamical and the second case corresponds to static fluctuations. 

To be simple and to be specific, let us consider (Xj)j>o, a stationary 
ergodic Markov process, with invariant measure p. Have in mind steady 
state descriptions of mesoscopic systems via Langevin dynamics, or 
stochastic lattice gases etc. By the ergodicity, for P— almost all states, 

(t) 

limxprp = p, with P the law of the process. For finite T it could 

be that p^^ very much resembles a particular probability measure p. 
Or, fixing an arbitrary probability law p, we can ask how likely it is 
to observe it as the statistics of {Xt)t. The theory of Donsker and 
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Varadhan [31 12] gives an expression of the rate function 

/.(/x) = -mf )_^/i(^)log^-— (2.2) 

x,y y\ J 

where Pr{x,y) are transition probabihties and the infimum is over all 
g > 0, appearing in the (logarithmic sense) limit T f +oo for fl2.ll) : 

P\pP ~ /i] ~ e-^^^'-^^\ T T +00 (2.3) 

Obviously, we can find the stationary distribution p from minimizing 
Jr(/i) over /i. That gives a variational characterization of p. Here is a 
special (limiting) case on which we will say more in Section |H Taking 
the limit r | corresponds to successive measurements with outcomes 
appearing with frequency 

1 /"^ 

Pt\x) = Pt{x) = - J Sxt=xdt 

i.e. a standard time-average. Then, 

P[Pt ~ /i] ~ e-"^'') (2.4) 

where I{p) is the Donsker- Varadhan functional that depends on the 
transition rates of the Markov process in the following sense: 

lip) = lim ^ = E ^(^) ^(^' ^) - ^ ^^ifr^(^' ^2-^) 

x,y ^ x,y 9^ ^ 

In fact, this is a special case of a more general formula valid far beyond 
the restriction to finite state processes, namely, 

I{p) = -mfEj^] (2.6) 

The expectation is over the process starting in the distribution p and 
Lg is the generator of the process acting on a function g (over which 
we vary in fl2.6p ). 

The variational principle originating from fl2.5l) - (l2.6p appears rather 
'unknown:' finding the stationary p amounts to finding the minimum 
of /(/i). Of course, even to evaluate I{p) already another variation 
must be done, that over g. We discuss more of that problem in Section 
m The next section deals with the opposite limit t ^ T of static 
fluctuations. 

3. Varying the relative entropy 

The limit r | +oo in the time-average fl2.ll) (with T f +oo) cor- 
responds to an infinite time separation between consecutive measure- 
ments. Then, 

lim /r(/i) = S{p I p) (3.1) 
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with S{fj, I p) the relative entropy. For finite state spaces (and supposing 
that fi{x) = whenever p{x) = 0), 

Sip\p) = J2pix)\og^ (3.2) 

We recognize the expression as it figures as thermodynamic entropy or 
another related thermodynamic potential in Gibbsian descriptions of 
equilibrium systems. 

The relative entropy (13. 2p also appears often in treatments of nonequi- 
librium systems. We present two ways of looking at that. 

3.1. Relative entropy and entropy production. It has often been 
emphasized that the relative entropy is a non- decreasing function for 
dissipative dynamics. Again, in our context of Markov processes on a 
finite state space, 

f^Sipt\p)<0 (3.3) 

for the time-evolved probability distribution pf, pt = p as it is the in- 
variant distribution. There is however need for more physical context 
if ever we want to interpret (13. 3p as the positivity of the entropy pro- 
duction. 

One possibihty goes as follows. 

The entropy production is the total change of the entropy in the 
world, system plus environment. Let us consider a transition x ^ y 
of the state of the system. We assume we have an entropy function 
5* for the system, and the change of the entropy in the system is thus 
S{y) — S{x). When the current between states x and y equals j{x, y) = 
—j{y,x) (possibly depending on time), then the average instantaneous 
entropy production rate for the system is | j, j(a;, y) [S{y) — S{x)\ . 
On the other hand, with some transitions x —>■ y in the system, one 
imagines there corresponds a fiow of particles or energy etc. to an exter- 
nal reservoir. These give rise to a change of entropy S{x, y) in the envi- 
ronment. Hence, the total entropy production is \ ^ j(a^, y) [S{x, y) + 
S{y) — S{x)]. Let us see how to model that with the limited means of 
a Markov process with transition rates k{x,y). 

When the system is described at time t by the probability distribu- 
tion pt, one can take as its entropy function S{x) = — log pt{x). The 
expected rate of change of the system's entropy is thus 

Ss = ^ Ptix)k{x, y) [ - log pt{y) + log pt{x)\ (3.4) 
x,y 

Further, since we are modeling an open system dynamics, the ratio of 
transition rates k{x, y) /kiy, x) should correspond to the entropy change 



5 



in the environment. E.g. if the states x and y carry different energies, 
H{x) 7^ H{y), then transitions x ^ y are only possible when the system 
exchanges energy with some heat bath. If the latter has inverse tem- 
perature P{x,y), then k{x,y)/k{y,x) = exp[—P{x,y){H{y) — H{x)}] 
and its logarithm gives the change of entropy in that reservoir. There- 
fore, under the distribution ^t, the expected rate of change of entropy 
in the environment is 



Se = ^/it(x)A;(x,?/)log 



k{x,y) 



x,y 



Upon adding (13.41) with (13.51) . we obtain the total mean entropy pro- 
duction rate in the distribution fit 

S = S, + Se = J2 f'tix)kix, y) log ^['^'^i^^i'^l (3.6) 
^ k{y,x)fit{y) 

which is always non-negative. 

Where do we see the relative entropy (13.21) in these expressions? When 
the stationary Markov dynamics satisfies the condition of (global) de- 
tailed balance, then 

k{y,x) p{x) 

and following (13. 6p . the rate of total entropy production then equals 



^ = ^ [l^tix)k{x,y) - ij,t{y)k{y,x)] log 



x,y 



p{x) 



Ed/it (x) ptix) /o 7\ 

x,y 

where in the second equality we have used the Master-equation for 
the Markov process. Only when there is detailed balance, we can in- 
terpret (13.21) as (the negative of) the 'entropy of the world', the time 
derivative of which coincides with the (total) entropy production rate. 
One way of dealing with that last remark is to consider only closed 
systems that are already incorporating various reservoirs and in which 
the (total) dynamics is detailed balance with respect to some overall 
equilibrium distribution. 



3.2. Relative entropy and the Hamilton-Jacobi equation. Let 

us assume that the Markov process has states in an interval of the form 
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[A/£,B/e\ C M and rates 

k{X,Y) = ^w{eX,Y-X) 

We have written capital letters to indicate that states X, Y represent 
values of a macroscopic observable(s), which is extensive in pro- 
portional to the (great) number of particles. A standard example is (a 
continuous time version of) the Ehrenfest urn model. There a rescaled 
realization is a; = eX e [0, 1], possible transitions r = Y — X = ±1, 
and the rescaled rates w{x, 1) = 1 — x, w{x, —1) = x. 

We consider the probability p{x, t) to have a value eX = x at time 
t. It satisfies the Master-equation 

e-^p{x,t) = — J dr w{x,r) p{x,t) + J dr w{x — er,r) p{x — er,t) 

(3.8) 

We will be interested in the stationary distribution p{x) verifying (13. 8p 
with zero left-hand side. Let us assume that 

pix) oc e-^(")/" (3.9) 

for some function S, up to leading order as e | 0. The entropy func- 
tion S describes static stationary fluctuations similarly as the relative 
entropy (13.21) does. To see that, observe that when starting the process 
at time — r from state a with S{a) = 0, then 

P[x{-t) = a, x(0) = 6] oc e'^^^)/", r j +oo (3.10) 
similar to computing (12.31) for T = r. 

There is an algorithm to directly compute S, obtained by rewrit- 
ing (13.81) as 

e-^p{x,t) = — J dr 1 — exp^— er^j w{x,r)p{x,t) (3.11) 

setting the left-hand side zero, and inserting formula (13. 9p into (13. lip 
with e I 0: 

drw(x, r) 1 — exp^r— S'(x) j =0 (3-12) 

We thus get a differential equation for the relative entropy! 

As an example, again with x G [0, 1], r = ±1, and w{x, 1) = 1 — x, 

w(a;, — 1) = x, we see the solution S{x) = x log2x + (l — x) log2(l — x), 

the relative entropy for a density x with respect to density 1/2 in coin 

tossing. 

The above scheme (with a somewhat different emphasis) was first 
mentioned in a paper of Kubo, Matsuo and Kitahara, [5J. We show 
how equation (I3.12p can be seen as a (stationary) Hamilton- Jacobi 
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equation. 

Formally, if we write 



ff(.,p)=/dru,(x.r)[l-expM] (3^3) 
then fl3.12p takes the form 

H(x,i-S)=0 (3.14) 



dx 

We have to explain yet that the expression (13.131) is indeed the Hamil- 
tonian corresponding to the path-space action. By that we mean that 
the probability to see the state x at time t when at time to the state 
was xq is given by the path integral 

P{x,t\xo,to) = I dcu exp — I ds L{x{s),x{s)) (3.15) 

•J XQ— >a; ^ J to 

where we 'integrate' over all paths u = (x(s); s G [to,t]) such that 
x(to) = Xq, x{t) = X, and the Lagrangian L{x, x) = px — H{x, p) is the 
Legendre transform of H. The reason for all that is the following. 
In equation (13.101) we compute the left-hand side by the path inte- 
gral (13.151) and assume that, as e | 0, the integral over all paths u can 
be approximated by the contribution of a single dominant path (which 
is the usual situation): 

P[x{—T)=a,x{0) = b]= I du; exp — f ds L{x{s),x{s)) 

Ja^b ^ J-T 

1 

■-inf / ds L{x{s),x{s)) 

e a-.b 



(3.16) 



Hence, 







5(6) = inf / dsL{x{s),x{s)) (3.17) 

which means that S is Hamilton's principal function for the Hamilton- 
ian H associated to the Lagrangian L. 

The idea of connecting the Hamilton- Jacobi equation with station- 
ary fluctuations first appeared in |5], and was reinvented by [Tj in the 
infinite-dimensional context. As we have shown above, these strategies 
amount to deriving a differential equation such as fl3.12p for the relative 
entropy (such as S in f l3.10p ) from the (path-space) Lagrangian (such 
as L in (EUD). 

4. Escape rates 

Recall the rate function f l2.4p -( l23|) for the fluctuations correspond- 
ing to time-averages with continuous measurements. In general, this 
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is not so simple to compute explicitly, unless the Markov process is 
detailed balanced. Indeed, one checks that in that case of detailed bal- 
ance with k{x,y)p{x) = k{y,x)p{y), the minimizing g in (12.51) is given 



We now give an application of that same formula (12.41) but for a 
nonequilibrium situation: to compute the escape rate from a local equi- 
librium. To be specific, we consider a well-known nonequilibrium model 
with just one particle (or, what is the same, independent random walk- 
ers) hopping on a ring with N sites on which work is done. The jumps 
are nearest neighbor but they are biased in the sense that the rate for 
jumping clockwise is k{x, x -|- 1) = p while the rate for jumping counter 
clockwise is k{x,x — 1) = q. It corresponds to an asymmetric random 
walk on a ring. The situation is sufficiently simple to understand that 
there is a unique stationary distribution p corresponding to the uni- 
form placement of the walker on the ring: p{x) = 1/N. The question 
is again about fluctuations: what is the rate at which the particle will 
escape from a given subset of the ring? 

It is most interesting to consider a connected subset A = {1, . . . , n} of 
n < N sites. We are given a probability distribution p localized on A, 
i.e., p{x) > for X G v4 and '^^^aI^^-'^) ~ laying it differently, there 
is no mass outside A. Following the fluctuation formula (12. 4p we want 
to find the rate I{p) in the probability that the particle remains in A 
with occupation weights according to p for the time T. 

To solve for that escape rate I{p) we must find the minimum in (12.51) . 
However since p is non-zero only on A, the only question is to minimize 
(over 5( > on {1, . . . , A^}) 



Obviously the minimizer satisfies g{x) = Q,x ^ A since all terms are 
non-negative anyway. So we are left with finding the minimal g for a 
functional I'{p) corresponding to the dynamics on A for which again 
k{x, X + 1) = p, k{x, X — 1) = q except when a; = 1, respectively x = n, 
in which case the particle can only hop to the right, respectively the 
left. That new dynamics on A is however an equilibrium dynamics 
(when indeed n < N). One checks the condition of detailed balance 
for the distribution pa{x) oc {p/qY,x G A. As a consequence we can 
take the minimizer g given as 
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and the rate function for /i becomes, exactly, for n < N, 

n-l 

/(/i) =p + q- 2 5Z + 1) 

a;=l 

n— 1 



(4.1) 



x=l 



The minimum of that /(/x) is reached for the uniform distribution 
= 1/n on A: we then have (14. ip = {y/p — y/q)"^ + '2y/pq/n. That is 
the inverse of the typical time that the particle feels the drive to escape 
from the set A. Fixing a time scale by putting pq = 1, we see that that 
escape time increases for p = q. 

Another application of formula (12. 4p concerns the relation with the 
so called minimum entropy production principle. Quite generally, the 
functional I{fi) is to first order in the driving and close to equilibrium 
an affine function of the entropy production rate. From that, we under- 
stand the minimum entropy production principle for Markov processes 
whose rates are a small perturbation from detailed balance rates, see 
[3l[7j. Details will follow in a separate pubhcation, [8j. 



5. Final comments 

The previous sections have given a short discussion of different types 
of nonequilibrium fluctuations. They vary from static to fully dynam- 
ical fluctuations. We have discussed a connection between the relative 
entropy as rate function for macroscopic fluctuations and a Hamilton- 
Jacobi equation. We have discussed a scheme to compute escape rates 
via the Donsker-Varadhan rate function for the dynamical fluctuations. 
Each time, we have seen relations with the entropy production. One 
final comment is in order however. 

Recent years have seen a revival of attempts of constructing nonequi- 
librium statistical mechanics. It has however not been clear how far 
we have been moving beyond close to equilibrium considerations. In 
particular, the obsession with entropy production is probably related 
to the good experience we have with the situation around equilibrium. 
For descriptions further away from equilibrium, one will also need to 
deal with fluctuations of time-symmetric observables (which are not 
in the sector governed by the entropy production), see also the dis- 
cussion in P, [9]. In fact, we expect that the rate functions that have 
been considered in the present paper all carry information about time- 
symmetric currents when these rate functions are explored beyond first 
order around equilibrium. 



10 



Acknowledgment 

K. N. is grateful to the Instituut voor Theoretische Fysica, K. U. Leu- 
ven for kind hospitality. He also acknowledges support from the project 
AVOZ10100520 in the Academy of Sciences of the Czech Repubhc. 

References 

[1] L. Bertini, A. De Sole, D. Gabrielli, G. Jona-Lasinio and C. Landim, Macro- 
scopic Fluctuation Theory for Stationary Non-Equilibrium States: J. Stat. 
Phys. 107:635-675 (2002). 

[2] A. Dembo and O. Zeitouni. Large Deviation Techniques and Applications. 
Jones and Barlett Publishers, Boston (1993). 

[3] M. D. Donsker, S. R. Varadhan. Asymptotic evaluation of certain Markov pro- 
cess expectations for large time, I. Comm. Pure Appl. Math., 28:1-47 (1975). 

[4] M. J. Klein and P. H. E. Meijer. Principle of minimum entropy production. 
Phys. Rev., 96:250-255 (1954). 

[5] R. Kubo, K. Matsuo and K. Kitahara: Fluctuation and Relaxation of 
Macrovariables, J. Stat. Phys. 9:51-95 (1973). 

[6] C. Maes. On the origin and the use of fluctuation relations for the entropy. 
Seminaire Poincare, 2:29-62 (2003), Eds. J. Dalibard, B. Duplantier and V. 
Rivasseau, Birkhauser (Basel). 

[7] C. Maes and K. Netocny. Time-reversal and entropy, J. Stat. Phys., 110: 269- 
310 (2003). 

[8] C. Maes and K. Netocny. Minimum entropy production principle from a dy- 
namical fluctuation law. In preparation. 

[9] C. Maes and M.H. van Wieren. Time-symmetric fluctuations in nonequilibrium 
systems. Phys. Rev. Letters, 96:240601 (2006). 



